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PREFACE 


As part of NASA’s responsibility to encourage and facilitate active exchange of 
information and ideas among members of the aviation community, an Aviation 
Safety/Automation workshop was organized and sponsored by the Flight Management 
Division of NASA Langley Research Center. The one-day workshop was held on 
October 10, 1989, at the Sheraton Beach Inn and Conference Center in Virginia Beach, 
Virginia. Participants were invited from industry, Government, and universities to discuss 
critical questions and issues concerning the rapid introduction and utilization of advanced 
computer-based technology into the flight deck and air traffic controller workstation 
environments. 

The workshop was attended by approximately 30 discipline experts, automation and 
human factors researchers, and research and development managers. The goal of the 
workshop was to address major issues identified by the NASA Aviation Safety/Automation 
Program. 

This report documents the results of the workshop. The ideas, thoughts, and 
concepts were developed by the workshop participants. The findings, however, have been 
synthesized into a final report primarily by the NASA researchers. The use of notes and 
taped recordings of the sessions facilitated this documentation. Many thoughts and 
concepts have been summarized so as not to identify any comment or thought with a 
particular individual or organization. This format was chosen to facilitate a free exchange 
of information and ideas at the workshop. 
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INTRODUCTION 


The National Aeronautics and Space Administration (NASA) has initiated the 
Aviation Safety/Automation Program to address the research and development of human- 
centered automation which would assist humans to attain increases in performance within 
the flight deck or ATC workstation, to monitor human performance, to detect and warn 
of human errors, and to manage contingencies. 

To encourage effective communication and the exchange of ideas with the aviation 
community, a workshop was conducted to address three major issues identified by the 
program. These issues were (1) the measurement and evaluation of situation awareness, 
intent, and performance; (2) functional validation of automated systems, especially 
Al/expert systems; and (3) successful completion of the program considering the need for 
technology transfer. The workshop’s participants were divided into three parallel sessions 
to address these questions, then brought back together in a plenary session. 

The Situation Awareness Group was asked to identify effective means of measuring 
and evaluating situation awareness, crew/controller understanding, anticipation, intent, and 
performance. They were also asked to consider complacency, vigilance, and boredom, and 
to identify voids in the research knowledge base. With the belief that increasing levels 
of technology and automation will be employed in the flight deck and controller 
workstations, the ability to define and evaluate the effects of automation on situation 
awareness has long been desired. 

The Functional Validation Group was asked to identify the level of functional 
validation of automation concepts/expert systems developed within the program that should 
be achieved. The methodology to obtain this level would also be identified. Guidelines 
and protocols are also needed to evaluate the performance and performance limits of 
knowledge-based systems. Methods are required to assure that an intelligent, automated 
system will not command an operating mode that could cause the system to deviate from 
its safe operating envelope. 

The Technology Transfer Group was asked to discuss effective means of technology 
transfer and how a successful completion of the program could be realized. Simulation 
and flight demonstrations of research results as a means of effective technology transfer 
were also discussed. It is desired that the research products from this program be utilized 
by the aviation community, so candid discussions of past successes and failures were 
encouraged. 
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Introduction 

The Situation Awareness group was asked to address the issues of measuring and 
assessing Situation Awareness, crew/controller understanding, anticipation, intent, and 
performance. With the belief that an increasing level of automation will be brought to 
the cockpit, it is important to be able to measure and assess these concepts so that 
meaningful evaluations of human interaction with specific automation concepts can be 
made. 


Background 

Historically, the term "Situation Awareness" primarily has been associated with 
attitude awareness, or, in the military, with knowing the relative positions, speeds, and 
directions of flight of friendly and hostile aircraft during a tactical battle. Today, however, 
Situation Awareness has taken on a much broader definition. 

Definition: Situation Awareness consists of two components: (1) the operator’s 

knowledge of the current physical and tactical environment; and (2) the process or strategy 
by which the operator switches attention to the multiple tasks being performed in order 
to keep that knowledge up to date. 

The specific physical and tactical aspects of Situation Awareness must be defined 
explicitly for the specific task environment. Generally, the physical environment will 
consist of X and Y position (longitude and latitude), altitude, aircraft attitude, heading, 
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speed, and other task-specific items. The tactical environment is highly task specific and 
could include such things as fuel remaining to fly to an alternate airport, height above 
ground, time to make a heading change, or altitude at which to level off. Situation 
Awareness is by its very nature an inferential process on the part ot the operator. It is 
not merely a collection of information coming in to the operator, but an active process 
by which the human operator infers concepts/models from the information coming in, 
organizes them, and combines them in various ways to perform necessary tasks. The 
process involves reducing the data down to as simple a level as possible. For instance, 
if the operator knows that the flaps are at a certain setting, the slats are extended, and 
the landing gear is down, then the airplane is in a "landing configuration. Generally, this 
is all that is needed; however, at other times more specific information may be needed. 

Situation Awareness should be considered to be a useful heuristic that allows us 
to focus on the different kinds of psychological activities and properties that are tractable 
to the researcher. Therefore, explicit definitions of these psychological activities or 
properties being investigated should be stated clearly in the research description. 


Operational Considerations 

The following operational aspects of Situation Awareness were identified: 

(1) Aircraft Path in Space (Navigation, Lateral Position, Vertical Position); 

(2) Aircraft State (Fuel, Sub-system, Configuration); 

(3) Position with Respect to Other Traffic (Terminal Area Operations, 
Sequencing on Airways); and 

(4) Communication (Air Traffic Control, Company, Passengers). 


Situation Awareness Issues 

The following issues were identified as critical: 

(1) What is a useful definition of Situation Awareness? 

(2) What procedures or techniques should be used in measuring Situation 
Awareness? 

(3) Should Situation Awareness be measured as an absolute quantity or 
as a change from a previous level? 

(4) Are there different types of Situation Awareness, and if so how does 
one know that the correct type of Situation Awareness is being 
measured? 
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(5) What is the nature of the relationship between Situation Awareness 
and Performance? 

(6) To what degree do Automatic systems give the operator feedback to 
enhance Situation Awareness? 

(7) When a simulation scenario is stopped to obtain subjective evaluation 
of Situation Awareness, can the simulation scenario be continued or 
should it be stopped, and a new scenario started? 


Possible Measurement and Evaluation Teehpkiu gs 

Crew/Vehicle Assessment 

Aircraft flight path deviation 
Frequency of control inputs 
Latency in achieving navigational goals 
Accuracy of following procedures 
Errors 

Response Time 

Subjective Responses (Questionnaire, Rating Scales) 

Workload Assessment (Physiological, Subjective, Performance) 

Visual Scan 

Performance History 
ASRS Database 
Error Rates 
Questionnaires/Surveys 
Flight path error 
Subjective Opinion 

Specific Situation Awareness Assessment Techniques 

Situation Awareness Global Assessment Technique (SAGAT) 

(Endsley, 1988) 

The following methods were identified for evaluating of specific situational 
awareness concepts: 

( 1 ) Problem solving research paradigms (Errors, Slips, Mistakes, Response 
Time Latency, Anomaly Detection Accuracy, Decision Making, 
Judgment); 

(2) Questionnaire (Quantitative, Situation Queries, "What If' Queries); 
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"Workload" Measurement Techniques (Physiological, Subjective, 
Performance); and 

Visual Scan (Eye Point-of-Regard). 


( 4 ) 


Crcw/Controller Understanding 

A definition of Crew/Controller Understanding was not agreed upon during ^the 
workshop, however, one possible definition was submitted prior to the workshop as "the 
extent to which any messages or instructions between the crew and the controller are 
understood as the sender meant to convey them". Crew/Controller interaction is one of 
the sources of Situation Awareness. It is analogous to other information sources, such 
as displays, aural cues, alerts, warnings, and information from other crew members. 

Many of the same measures used for Situation Awareness could be applied to the 
Crew/Controller Understanding Issue. In addition, the following measures should be 
considered: 

(1) Compliance with Commands; 

(2) Response Latency; 

(3) Accuracy in Interpreting Messages; 

(4) Types of Errors; 

(5) Information Transfer Rate; and 

(6) Questionnaire. 


Anticipation 

To some extent, the degree to which a pilot or other operator can anticipate and 
prepare for seemingly unexpected events is another measure of Situation Awareness. 
Two types of activities were discussed which help the crew anticipate events: preparatory 
processes and communication. In a preparatory process, the crewmembers anticipate 
their own needs and prepare for those future needs either overtly or covertly. Most forms 
of training are overt, while mental rehearsal, in which a crewmember uses periods of 
relative inactivity to mentally rehearse upcoming tasks and to consider possible 
contingencies, is covert. A serious research problem remains in that it is very difficult to 
detect and quantify covert preparatory processes, which actually increase "off-peak" 
workload temporarily, and to correlate these processes to better task performance. 
Communications, either with or between other aircraft and Air Traffic Control (ATC), can 
also help the crew anticipate future workload demands and prepare properly. It was 
mentioned that the use of DataLink may have an impact on this source of information. 


6 


Imam 


With regard to system knowledge of operator intent, it was proposed during the 
workshop discussion that intent inferencing is best described as maintaining or improving 
the Situation Awareness of the aircraft systems and automation. For the automation to 
effectively carry out its assigned duties, it needs to be aware of much of the same 
information as the operator. In addition, it needs to be aware of what the operator is 
doing now, how this fits into his overall goal, and what he plans to do in the future. 
While this level of understanding does not exist in current cockpits, its development may 
be necessary to provide the forms of advanced decision aiding envisioned by many. A 
simpler form of intent inferencing may be possible using eye point-of-regard, so that the 
system can provide the information it determines the crewmember is seeking. 

It is also interesting to note that pilots in current-generation aircraft are doing 
intent inferencing themselves with respect to the automation. Questions such as: "What 
is the system doing now?" "What is it going to do next?" "Why is it doing that?" are 
frequently cited as examples of the difficulty of maintaining good Situation Awareness. 


Summary 

A working definition of Situation Awareness was presented in which Situation 
Awareness was conceptualized as consisting of two components: (1) the operator’s 
knowledge of the current physical and tactical environment; and (2) the process or strategy 
by which the operator switches attention to the multiple tasks being performed in order 
to keep that knowledge up to date. Because of the inferential nature of Situation 
Awareness, it cannot be quantified directly, and must be derived through the measurement 
of other parameters. A number of the possible measurement and evaluation techniques 
were presented. 
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Gary Lium, FAA 

Bill Miles, Douglas Aircraft Company 
Everett Palmer, NASA/Ames Research Center 
Wendell Ricks, NASA/Langley Research Center 


Introduction 

The Functional Validation Group was asked to address issues concerning the level 
of validation (functional only) that should be achieved in the Aviation Safety/Automation 
Program for the new automation concepts produced. 

The remainder of this report describes the scope of the specific issues that were 
addressed by the working group, the stakeholders in the validation process for the 
Aviation Safety/Automation Program, validation as consensus, and the evaluation process 
for new automation concepts. 


Definition of Functional Validation and Scope of the Working Group 

Validation is an extensive process. It involves defining the requirements for a given 
problem, and from those requirements deriving a solution. One concern is determining 
whether the correct problem is being solved; i.e., whether the requirements are correct 
for the problem. Verification is the process of determining whether the solution concept 
satisfies the requirements. Although this group was tasked to address the entire validation 
process, the entire process was considered too large to address in the given amount of 
time. The group decided, therefore, to focus on the question of verification: assuming that 
the set of requirements is given, how should the solution concept be verified? Two aspects 
of verification for concepts, such as the ones developed in the Aviation Safety/Automation 
Program, are research evaluation (often involving human-in-the-loop experiments) and 
FAA certification. The outputs of the evaluation are important inputs to the certification 
process. Although the group discussed certification, the emphasis was placed on the 
evaluation process. The group also decided that consideration of the evaluation of 
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advanced automation concepts was more appropriate, rather than a particular technology 
such as artificial intelligence. 


The Stakeholders 

The group decided that identification of the stakeholders in the validation of 
Program outputs was important. The outputs of the Program are: (1) guidelines; (2) early 
warnings of technology considerations; (3) technologies and concepts; and 
(4) methodologies for validating those technologies and concepts. 

The stakeholders in the validation of Program outputs were determined to be 
NASA, airframe manufacturers, airlines, the FAA, and the public. The lines of 
communications among the stakeholders are depicted in Figure 1. 



Figure 1. Aviation Safety/Automation Program Stakeholders and Lines of Communication. 


Validation As Consensus 

The group observed that the current approach to validation of new automation is 
a consensus-building process, rather than a formal proof, because formal proofs are not 
possible Therefore, regardless of the approach taken to show that a concept meets the 
requirements, the hope will be for the relevant stakeholders to conclude with a consensus. 

The consensus is developed iteratively. First the developers must convince 
themselves that the concept is satisfactory. Sometimes this can be done by analytical 
means, and sometimes just by interacting with the concept. This ability to use interaction 
as an alternative is important because analytical means are not always available. 1 he next 
step usually involves broadening the circle of consensus to include other stakeholders, this 
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process continues until all relevant stakeholders are satisfied that the concept is viable. 
During this process, it is often realized that the concept has limitations. These limitations 
are noted, but there is often enough consensus to proceed with the process. This is very 
different from a proof. 


Ey^uatimiJ^rocess 

The evaluation process is iterative, as shown in Figure 2. Functional validation does 
not occur at any one point in the process. Rather, functional validation is the entire 
process of formulating the problem to determine requirements, developing a solution, and 
gaining experience with the solution concept. Some of the initial experience is gained 
while developing the initial solution concept. Ongoing experience is gained during the 
entire evaluation process. This type of evaluation process may be even more important 
for concepts that use artificial intelligence (AI) technology, because suitable analytical 
evaluation techniques are often lacking. 

The question remains of how much of this evaluation cycle is necessary before the 
solution concepts can be transitioned to the user. Addressing this question is the research 
goal of full-mission simulation tests in which the research concept is compared to some 
baseline. It was strongly noted that part-task simulation is a very important step but is not 
sufficient. Although part-task simulations are important, full-mission simulation is necessary 
to test the interaction of the concept with other parts of the overall system. This 
interaction and integration with other parts of the system is particularly important to the 
FAA, because a major focus of that agency is to examine how a new concept interacts 
with the existing system. 



Figure 2. Evaluation Process. 
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Although full-mission simulation was agreed upon as a minimum for functional 
validation, not all the workshop participants agreed that it represented the end of the 
evaluation process. No consensus was reached among the group members regarding the 
need for flight test. Flight tests were viewed as increasing the confidence in some concepts, 
while being of little help in others. For example, a flight test could not be conducted in 
which an actual fault was introduced to test a fault management system such as 
Faultfinder. The group agreed that it is important to remember that flight tests are not 
intended to prove the absence of errors in a system, but rather are intended to increase 
confidence in a system. For some solution concepts, flight tests may be necessary to gain 
the desired level of confidence. 

A very important point made in the working group was the importance of involving 
the potential customers (primarily the airframe manufacturers) very early in the research 
process. It was considered inappropriate to fully develop and evaluate a concept, and then 
expect the users to incorporate it into their systems. The problem that arises is that the 
organization will not be familiar with the concept and will not understand its capabilities 
and limitations. It was believed to require a great deal of effort to transfer a concept if 
there is no early involvement. 

The utility of the experience gained in the evaluation process relies heavily on 
scenarios. It was observed that it is important to remember that as scenario complexity 
increases, the cost associated with developing and using the scenarios increases. Cost also 
increases as the set of scenarios is enhanced to achieve increased coverage. Therefore, 
there is often a tradeoff between scenario complexity and coverage. Often the evaluation 
is designed to a fixed cost level, so some compromises may be necessary. If the concept 
is determined to be viable in this limited evaluation, it is then placed in an environment 
where it will receive ongoing use. This use contributes to the ongoing validation of a 
solution. 


Summary 

The working group identified the stakeholders in the validation of concepts and 
technology produced by the Program. These stakeholders included the airframe 
manufacturers, the FAA, the airlines, and the public. The group agreed that the current 
approach to validating automation is a consensus building process, rather than a formal 
proof of correctness, because a formal proof is not possible. It was agreed that a 
consensus building process will also apply to functional validation of the Program concepts, 
and that some of this will occur during the evaluation of the concepts. The evaluation will 
be an iterative process, and should include taking concepts to full-mission simulation at 
least. The group noted it is highly desirable for the potential customers of the Program 
concepts, especially the airframe manufacturers, to be involved early in the research and 
evaluation process The group also observed that the utility of experience gained in the 
evaluation process relies heavily on the scenarios used. 
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Compiled by 

Renate Roske-Hofstrand and Paul Schutte 
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Background 

The Program Success and Technology Transfer group was asked to address the 
issue of defining means by which to assess the success of the program and to enhance 
technology transfer. 


The group discussed the fact that there are basically two ways of measuring the 
success of research programs. One is through assessing the impact of research products 
and the other is through assessing the quality of the work being performed. Implicit in 
the concept "technology transfer" is the assumption that a program’s life-cycle is more or 
less linear and that the various program products can impact current problems in aviation 
safety as well as influence future generation aircraft and systems designs. The 
interdependency between program success and technology transfer is complex and the 
timing of the assessment is crucial for arriving at a sought-after answer. For example, it 
may well be that we can get different answers to the question, "are we successful ?", one 
year from now, at the end of the program’s budget cycle, or ten or more years from 
today. In addition to these maturational considerations, the various types of program 
products may call for differential treatment in terms of their relative suitability for transfer 
(or measurement of success). 

The group suggested that the most effective means for accomplishing the transfer 
of research products is directly tied to the degree of movement of people and information 
among and between industry and government agencies in this country. Several specific 
solution paths for successful technology transfer were discussed in light of a cursory 
analysis of current impediments. These are detailed in a later section. 
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What Gets Transferred? 


Traditional transfer vehicles, such as NASA Technical Memoranda or other 
journal-type reports constitute a traditional style of information transfer but were judged 
by this group as relatively unsuccessful in terms of having near term real-world impact. 
More recently, software tools, when used and applied by NASA "customers" to their own 
R & D efforts, are becoming an immediately sought after product that can be easily 
disseminated to the relevant players in the community. These products can have an 
important impact upon future developments. 

Fully engineered systems and designs as end-products are most closely linked to the 
term "technology", but the working group felt that, in the human factors arena at least, 
systems constitute more of a methodology embodied in a set of tools rather than unitary 
end-products that can be adopted "as is" for use in the operational aviation community. 
In fact, human factors contributions to the research and development store of accumulated 
system knowledge (simulations were included in the discussion) is by nature difficult to 
assess and transfer ("when the best human factors work is done nobody notices, only when 
problems arise, i.e., something goes wrong, are human factors people called upon"). 

A primary way to forestall these adverse forces is to ensure that human factors 
professionals are an integral part of the research and development process. 
Interdisciplinary working teams made up of representative experts from both government 
and industry were thought to be the optimal vehicle for accomplishing effective technology 
transfer. 


Im pediments to Technology Transfer 

Several impediments to technology transfer exist, including economic constraints, 
socio-historical factors, and domain-inherent limitations that hinder rapid, transfer progress. 

Economic Forces. — Like other companies, the airline industry is basically 
economically motivated. Manufacturers will build what their customers, the airlines, 
demand. Different customers will pose different requirements since the cause-effect link 
between good design and accident rate is so precarious. In the absence of hard data 
derived from comparing "design A" with "design B" in terms of their respective potential 
for producing differential human error rates, the customer will state his requirements 
based on cost, subjective preference and/or tradition, as well as other, hard-to-track, 
criteria. NASA must therefore invest time in educating all the customers, especially the 
airlines. 

Criticality of a potential error is also an important consideration in the success of 
new designs. For example, "tuning radio frequencies" was mentioned to illustrate that 
mistakes are made all the time, but that the potential cost of a redesign in this domain 
far outweighs the perceived benefit. The economic impact of design on pilot training or 
training of maintenance personnel deserves attention. A design idea may not be able to 
be "sold" based on good human factors principles; however, if one can show that savings 
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may be achieved in the training area with the particular design, the idea will be popular 
and likely to be adopted. 

Measurability. — A measurement problem exists when discussing human factors 
variables. For example, which features in an avionics display lead to incidents/accidents 
in contrast to those which are simply annoying? It is also difficult to get accurate baseline 
measures on everyday mistakes in the cockpit. Pilots are generally very good at catching 
most of the errors that result from suboptimal design. In general, if one can show (in 
high fidelity situations) that a feature will reduce the number of incidents or errors, this 
constitutes sufficiently compelling evidence to convince both the customer and the 
manufacturer that this is something they want and need. On the other hand, a simple 
"study" based on some "academic" theoretical considerations is likely to be ignored by the 
aviation community. 

Research and measurement in the training process was also suggested as an indirect 
vehicle for measuring the "goodness" of automation. Several U.S. carriers are following 
the trend started by their European counterparts of hiring ab-initio-pilots out of necessity. 
If a type of design can help a pilot with little experience make fewer errors, then the 
automation design will be deemed successful by the airlines. In other words, cockpit 
human factors must be tailored to address issues that will reduce training requirements 
(hence cost) for the inexperienced pilot. 

Socio-Historical Forces. — Research and development in the aviation community 
is still largely driven and dominated by the aerodynamics and structural engineering 
contingent of the airframe manufacturers and not by the end user, the pilot. Only 
recently has there been a perceived shift among customers benefiting the avionics side. 
Next generation aircraft are still being specified primarily in terms of structures and 
propulsion and not in terms of crew performance and cockpit design. If early human 
factors considerations arise, they are certainly heavily outweighed and overshadowed by 
their more influential "metal-bending" counterparts. 

One factor that may seriously affect human factors efforts and the Aviation Safety 
Automation program in the next several years is public opinion. It was pointed out that, 
at the current rate of growth in the airline industry, the worldwide accident rate, in terms 
of sheer numbers perceived by the public in only 5 years time, may be as high as one 
major accident every 3 weeks. The present day culprit "human error" will continue to 
arise as a causal factor, and public outcry may demand more "human factors" 
consideration. 

Tvpe-Rating and Training. - The airlines presumably want their pilots to have type 
ratings that cover all the aircraft in their fleets. Yet, airlines will resist new avionics 
technology if it increases the likelihood of errors when pilots transition back to older 
technology aircraft. This line of reasoning was felt to be the case even if a "better 
mousetrap" was available. If there were an increased effort to standardize aircraft types 
(by manufacturer at least), then there would be an improvement in the ease of 
transferring technology. 
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Incompatibility with Ground Control. - Another major impediment to continued 
progress in cockpit technology is the incompatibility between the sophistication of available 
cockpit technology and the technology used by ATC. This mismatch is thought to be 
responsible for defeating much of the potential benefits derived from innovative cockpit 
research such as advanced flight management systems concepts. 

Proprietary Information. — In order for innovative ideas and methodologies to 
tlourish it is important to exchange information freely. A major stumbling block to the 
free flow of information between government research labs and the aviation industry 
(avionic and airframe manufacturers) is the proprietary nature of advanced information. 
Mutual benefit can be derived if the manufacturer allows government personnel access to 
their research labs and NASA Invites manufacturer personnel to participate in their 
research endeavors. This interchange would go a long way toward developing mutual 
education and training and ensure relevancy. 


Strategies for Technology Transfer 

A list of potential research products is given in Table 1. These should have an 
impact on improved training and accident prevention. It should also be noted that the 
transfer of products or methodology can each have an impact on air transport operations. 
Products accomplish this simply by being adopted, whereas methods and data influence 
how future technology actually gets developed. 


Table 1: Program Products to be Transferred 


CATEGORY 

SPECIFIC ITEMS 

Information 

Tools, Measures, Data, Tactical Support 

Technology 

Systems, Designs, Hardware 

Methods & Methodologies 

Reports, Software, Expert Systems 

Guidelines 

Training, Operational Design 

Candidate Designs 

Early Prototypes 

Technical Support 

Expertise through People Exchange 


Training Impact . — The research products that have an impact on training are very 
important to the airlines because a reduction in training due to automation translates 
directly into cost savings. 

Accident Prevention . — Accident prevention is another important area since it is 
a concrete issue and usually gets the backing of everyone: the manufacturers, the airlines, 
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and the pilots. It is therefore recommended that human factors research should always 
strive to demonstrate improvements and/or recovery from known accident cases. 

A related topic addresses the available research infrastructure. The aviation 
community should strive to standardize scenarios used in the evaluation and testing of new 
ideas, equipment and procedures. If this element in the research environment were fully 
intact and scenarios could be widely shared by the various research houses, large strides 
in obtaining relevant relatable data could be made. The availability to any researcher of 
simulators with standardized operational scenarios based on critical accident data to test 
technology issues is vital to having an impact on safety. 

Early Involvement of People . — In order to ensure that research products reach 
their destination it is important to explicitly define NASA’s potential customers. The 
research effort must involve these people at all phases to ensure technology transfer. 
Involvement of these people in the research effort allows immediate feedback as well as 
immediate dissemination of results. It also develops an advocacy group to bring 
knowledge to the uninformed and reluctant. Table 2 lists the potential technology transfer 
points in the aviation research arena. 


Table 2: Technology Transfer Points 


Transport Aircraft Manufacturers (domestic and foreign) 
Business Aircraft Manufacturers 
Avionics Manufacturers 
Airlines (domestic and foreign) 

Pilots (ALP A, APA) 

Transport Associations (AT A I AT A) 

Controllers (NATCO, ATCA) 

Standards, Regulation, Certification (FAA,CAA,etc.) 
Research Community (Academic and Industrial Standards) 
Military Aviation (DOD) 

Safety Boards (NTSB) 


Past successes in human factors technology transfer have resulted from close 
cooperation between researchers and the airline pilots or the airlines, i.e., the end user. 

"Standardize" Research Among Centers and Industry . — Besides facilitating effective 
dialogue, the development of standardized, compatible facilities and jointly developed 
scenarios and methodologies will greatly enhance productivity. A tool kit of various 
simulation modules, scenarios, and data collection tools can be assembled by NASA and 
others for exchange so that research methodology does not need to be re-invented every 
time an issue is to be investigated. At the moment considerable research energy is spent 
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on getting the necessary resources up and running rather than dealing with solutions to 
key issues. 

Joint use research facilities with very high fidelity, two-crew, flight deck operations 
in realistically complex ATC simulations are needed to gather data for research with 
actual line pilots. In order to make a substantial impact on the aviation industry a need 
exists to establish an entirely new way of working together. NASA can facilitate the 
development and cooperation of joint working teams among industry and government. 
Such cooperation would be a very effective way to exchange information in a timely and 
informed manner and will result in accomplishing a large amount of necessary work in the 
shortest possible timeframe. 

Joint National Research Teams. — Working in the framework of a national research 
unit is of benefit to all participants since better ideas are generated, people get fully 
trained, and investigators will take lessons learned back to their own institutions and 
companies. This model is borrowed from the Japanese but may be successfully applied 
to our own situation in the United States on any project or program with national scope. 

Table 3 shows the proposed distribution of personnel during a program’s life cycle. 
During the problem definition phase it is important to involve all interested parties. 
Solutions should be proposed by a smaller interdisciplinary group of experts and 
prototyping and testing should occur by a selected group of experts (solution authors) in 
a consortium-type contract arrangement. 


Table 3: Process for NAS Technology and Transfer 


INVOLVEMENT 

PROCESS STEP 
SELECTED 

ALL 

PARTICIPANTS 

SMALL 

GROUP 

TEAM 

NASA AND 

Problem Definition 

X 



INDUSTRY 






Propose Solution 


X 



Prototype & Test 



X 


Lessons Learned 





Technical Support 



X 

INDUSTRY 

Application of Solution 


X 



The main message of this research model is that technology simply can not be 
planned from point A to point B, rather it is people who provide the critical element for 
making technology transfer happen. A well defined government/industry program with 
clear problem definitions can have a much bigger impact than just another program plan 
based on past government-industry relationships. 
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Other Suggestions 


Group discussions included a host of other related issues. These can be categorized 
primarily into necessary preconditions for success of the program and specific suggestions 
for dealing with currently perceived shortcomings. Five global preconditions are listed 
in Table 4. 


Table 4: Preconditions for Program Success 


Clear and Commonly Agreed Upon Goal Statements 
Economic Incentives (Industry) and Motivation (Problem Criticality) 
Measurement Technology (Standardized Environment Must Exist) 
Ease of Interaction (Modern Media and Management Support) 
Stable Funding (Commitment by All Parties) 


Simulator Working Group. — Several points of confusion exist on how to maximize 
the value of simulation research. Standard methodologies are lacking and there is a sense 
that we could obtain a lot more information out of simulator research than we are 
presently achieving. A working group should be initiated to discuss these issues and to 
look at the various simulation projects and facilities, their respective capabilities and 
methodology. This working group should arrive at guidelines for achieving a high degree 
of standardization and compatibility between government and industry research facilities. 

Airline Focus Meetings. — The perception of various NASA outsiders is that there 
is a noticeable gap between the research being performed and the operational aspect of 
transport aviation. To overcome this perception it is suggested that NASA hold regular 
meetings with the airlines as a focal point where the researcher-participants’ task is to 
really understand what the airlines are saying about the various operational problems. 
The airlines would identify and prioritize their issues and explain to the researchers the 
operational problems they are facing. The recent ATA National Plan To Enhance 
Aviation Safety Through Human Factors represents a positive step in this direction. The 
two points emphasized in relation to this suggestion were that (a) the technical researchers 
should participate and (b) there should be a thorough sampling of airline participants, 
not just a rotation of a small select group. 

Information Exchange. - Several means for enhancing the information exchange 
infrastructure were discussed. Specific suggestions to stimulate heightened information 
exchange included the publication of a newsletter with joint contributions from industry 
and government researchers working on a specific problem (having one specific mandate). 
An electronic version of this newsletter could also be made available. Innovations such 
as hypermedia could serve to make demonstrations and news items an extremely 
stimulating and interesting forum. 
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The general notion of portability of software and hardware was also mentioned as 
a potential enhancement to information exchange and, therefore, technology transfer. We 
basically need to find better ways to share things quickly. "Share-ability" of information 
in the applied human factors domain is somewhat problematic since industry has 
proprietary concerns and many researchers outside specific avionics or manufacturing 
companies are not sufficiently informed about plans for next generation technology. What 
is basically needed is information access, compatibility and portability of software 
("software speaks louder than words") and hardware to achieve a spirit of cooperation. 
The present state of affairs can be characterized as a general willingness to cooperate but 
lacking the necessary focus and some vital pieces of the necessary infrastructure. 

Asking for Feedback. — One of the most obvious but often neglected ways of 
measuring success is to ask for feedback directly. NASA could seek critical feedback by 
exposing research ideas to the aviation industry. While the current structure of the 
Aeronautical Advisory Committee does some of this, it is primarily program oriented and 
therefore does not provide specific technical critique desired by the researchers. 

One possible vehicle for this kind of feedback is for NASA to send out survey 
questionnaires on a fairly regular basis asking for opinions from industry on how much 
impact NASA-sponsored work has had on them or how helpful NASA reports have been. 
Technology transfer in this sense becomes clearly a two-way path. 


Summary 

Human Factors Research by its nature is more difficult to track and evaluate than 
its engineering and technology counterparts. Multiple products exist that are candidates 
for "transfer", these range from avionics system concepts and designs to research 
methodologies and software tools. Progress on all of these fronts will result in a 
cumulative success path, i.e., have immediate and eventual impact on the National 
Airspace System (NAS). To enhance this process it is important to be flexible and 
responsive to voices from the operational community. People and expertise should be 
developed along with technology. NASA can play an important role in facilitating free 
information exchange and a spirit of cooperation that will help create the appropriate 
climate necessary for technology transfer to occur. 
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SUMMARY AND CONCLUSIONS 


A one-day workshop was held to address three major issues identified by the 
Aviation Safety/Automation Program. Three groups were formed to address each issue. 
The first issue concerned the measurement of situation awareness. It was determined that 
situation awareness can not be measured directly, but must be inferred from parameters 
that are measuring human performance. The development of test scenarios that explicitly 
address the level of situation awareness in question was deemed to be very important. 

The second issue concerned the functional validation of automation technologies 
and concepts. It was agreed that a consensus-building process should be applied to 
automation concepts developed within the Aviation Safety/Automation Program, and that 
some of this will occur during the evaluation of the concepts. The evaluation will be an 
iterative process and should include taking the concepts to full-mission simulation at least. 
It was also agreed that the experience gained in the evaluation process relied heavily on 
the quality of the test scenarios. 

The third issue concerned the transfer of research products to the aviation 
community. It was determined that the research products take various forms, including 
information, tools, technology, methods, guidelines, and candidate designs. Several 
impediments to technology transfer of human factors research products were identified; 
economic forces, ability to measure safety benefits, historical forces 
(aero/propulsion/structures disciplines have been and are the major drivers). The group 
noted that past successes in human factors technology transfer have occurred when close 
cooperation existed between researchers and the airframers/airlines. 
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APPENDIX A 


SITUATION AWARENESS-CRITICAL PHENOMENON OR ILL-DEFINED CONCEPT? 

Nadine B. Sarter and David D. Woods 
Cognitive Systems Engineering Laboratory 
Ohio State University 


Situation awareness is supposed to be an essential prerequisite for the safe 
operation of complex dynamic systems. Especially in the aviation domain, the term has 
become one of the most fashionable performance-related concepts since recent accidents 
and incidents have triggered concerns about the potentially disadvantageous effects of 
cockpit automation on situation awareness. This problem is being addressed by major 
research plans (e.g., National Plan to Enhance Aviation Safety through Human Factors 
Improvements, ATA, 1989; Aviation Safety/Automation Research Plan by NASA, 1988) 
and reflected by research findings like the following (adapted from Wiener, 1989): 


In the B-757 automation, there are stil 
things that happen that surprise me. 
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One of the major reasons for the impact of automation on situation awareness is 
seen in the associated changes of level and timing of feedback. Whereas manually flying 
a plane provides immediate low-level feedback of the consequences of each input, 
automated systems involve longer time-constant feedback loops. Pilots are informed about 
the system’s state and behavior at a very high level of overall outcome of a variety of 
activities. Maintaining awareness becomes difficult as the evolution of situations and the 
overall mission is no longer perfectly transparent. 

Despite the concept’s popularity, a widely accepted definition of situation awareness 
is still missing. Processes involved in achieving and maintaining it are not yet completely 
understood. Surprisingly enough, research is already going on that aims at improving 
situation awareness by designing new cockpit systems and displays. 


This paper is an attempt to fill the above-mentioned gaps by (a) looking at the 
cognitive basis of situation awareness' (b) giving special consideration to the meaning of 
the term "awareness", which is critical for developing appropriate assessment and 
manipulation techniques; and (c) suggesting possible new approaches to the investigation 
of situation awareness. 


The Concept of "Situation Awareness" 

Both the commercial and the military aviation community have proposed a variety 
of definitions of situation awareness. The following list presents some examples to illustrate 
their differences in abstraction level and degree of comprehension. Those aspects referring 
to the definition of "awareness" are underlined. 

(1) "...the pilot’s knowledge about his surroundings in light of his mission’s 
goals..." (Whitaker and Klein, 1988) 

(2) "...It means that the pilot has an integrated understanding of factors that will 
contribute to the safe flying of the aircraft under normal or non-normal 
conditions..." (Regal et al., 1988) 

(3) "...the perception of the elements in the environment within 3 volume. of 
time and space, the comprehension of their meaning , and the projection of 
their status in the near future ..." (Endsley, 1988) 

(4) "...where: spatial awareness (the pilot’s knowledge of his location in space 
and of the spatial relationship between objects) 

what: identity awareness (the pilot’s knowledge of the presence of threats 
and their objectives and the pilot’s awareness of system state variables) 

who: responsibility or automation awareness ( knowledge of who is in 
charge) 
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when: temporal awareness ^ knowledge of the occurrence of events as the 
mission evolves)..." (Harwood et al., 1988) 

(5) "...the pilot’s knowledge of: 

(a) where both friendly and enemy aircraft are and what they are 
doing 

(b) what the pilot’s flight knows and what the flight’s options are 
for offense and defense 

(c) what other flights know and what their intentions are 

(d) what information from above is missing..." (McKinnon et al., 
1986) 

These definitions, as well as most others in the aviation literature, basically address 
two issues: (a) the essential components and (b) the temporal dimension of situation 
awareness. 


Components of Situation Awareness 

Pilots need to pay attention to a large amount of data from a variety of sources 
in their environment such as cockpit instruments, other crew members, and the plane’s 
vicinity. Many definitions of situation awareness try to define in detail which part of this 
information is critical for the pilot. Such efforts suffer, however, from the fact that, in 
dynamic environments, the relevance of data depends on their context and will therefore 
vary within and between flights "as a function of specific task, the environment, and the 
tactical objective" (Hamilton, 1987). 

Rather than specifying the essential components of situation awareness, it seems 
more important to understand what factors may affect the acquisition and processing of 
data in general and how they can be counteracted. Factors that would deserve further 
consideration in the aviation context are, for example, lowered alertness as a consequence 
of vigilance decrements and fatigue or the effects of time pressure: 

— Vigilance decrements are known to occur after as little as 20 minutes into 
a task and may cause such problems as narrowing of perceptual focus or an 
increasing number of attention lapses that in turn result in missing data. 
However, it is not yet clear whether this concept is only true in the context 
of simple monitoring tasks or whether it also holds for complex environments 
like the flight deck. 

— Fatigue can result in, for example, the loss of cohesive perception; i.e., the 
pilot perceives all relevant data one by one but can not integrate them to 
an overall picture of the situation. With increasing fatigue, the pilot may 
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also have difficulty in accurately recalling previous flight events when 
required. 

While most definitions of situation awareness only talk about the need to be aware 
of available data it can be as fatal not to notice that some information is missing. If the 
pilot erroneously assumes that he is on track he may run into trouble if he actually needs 
missing information in an emergency situation when time constraints do not allow him to 
search for it. 


Temporal Dimension of Situation Awareness 

In dynamic environments like the flight deck, minor deviations or failures that are 
uncritical in themselves may evolve and interact over time to become a major threat. 
Therefore, it is important to observe them, keep them in mind, and integrate them over 
time. Recurrent self-contained and isolated situation assessments are not sufficient. The 
goal must be an active model of the world that is continuously updated in accordance with 
ongoing events and trends. Temporal awareness is important for understanding and 
solving problems that are caused and influenced by precursors in the past. It is also 
essential for the prognosis and prevention of potential problems in the future on the basis 
of available data (see Endsley, 1988; Harwood et al, 1988). The following figure sketches 
some scenarios that require temporal awareness in order to cope with an existing or a 
future problem: 



PAST 

PRESENT 

PILOT 

FUTURE 

(1) 

PREC(S) _ ■> PROBLEM Al 

PROBLEM A2 

— DIAGNOSIS 

(2) 

PREC 

PREC — ► PROBLEM 

PREC — 

(3) 

(4) 

(5) 

(6) 

PREC A ► PROBLEM Al 

PROBLEM A2 

PREC B ► PROBLEM B1 

PREC(S) Crucial PREC — ► PROBLEM 

PREVENTION — ► 

PREC(s) ► PROBLEM 

PREC — 

PREC 

PREC 

► PROBLEM 


(PREC-Precursor) 
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As indicated by the figure, problems can evolve in a variety of ways, such as: 

A single problem may be caused by several precursors, occurring either at the same 
time (case 5) or one after the other (cases 2 and 6). One single precursor can 
result in multiple problems (case 1). A variety of precursors may not be critical in 
themselves but cause a problem if one additional crucial event happens (case 4). 
Or the pilot may be confronted with a conglomeration of problems related to 
different precursors in the past (case 3). 

Coping with one major, instant-onset, notified failure has been investigated to some 
extent and is part of every pilot training. The temporal aspect of more subtle and complex 
scenarios, however, has been largely neglected up to now. Incident reports and discussions 
with airline pilots indicate that this aspect does not correspond with its importance for 
aviation safety. Thus, it will be important to look at the risk potentials of the outlined 
scenarios. 


Cognitive Processes Underlying Situation Awareness 

While many authors try to be specific in determining the important components of 
a "situation", they avoid explicitly defining the term "awareness" which is often simply used 
synonymously with expressions like "knowledge", "integrated understanding", or "internal 
model of the world" (Endsley, 1988). A more thorough analysis of the phenomenon is 
necessary for its investigation and manipulation. 

Situation awareness is based on recurrent situation assessments and the integration 
of their results over time. Situation assessments have been defined as "a complex process 
of perception and pattern matching greatly limited by working-memory and attentional 
capacity" (Endsley, 1988). This process is supposed to involve three different levels: 

Level I Perception of situational elements 

Level II Based upon schemata, i.e., knowledge structures stored 

in long-term memory (Rumelhart, 1984), which are 
activated by recognized patterns in incoming data, the 
pilot forms a holistic picture of the situation in working 
memory (Wickens, 1984; Fracker, 1988). This picture 
determines his further information search and attention 
allocation (Hayes et al., 1977). 

These processes require a high level of expertise which 
has been defined as "highly developed repertoires of 
pattern-oriented representations" (Chase and Simon, 
1973). 
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Level III Projection of future status and actions of situational 

components This level is commonly referred to as 
"being ahead of the plane", which— from an aviation 
safety point of view— should be the main objective of all 
research and training efforts. 

A variety of problems affecting information processing (e.g., distractions, biases) can 
occur at all levels and would result in inadequate or incomplete situation assessments. But 
even if the pilot manages to overcome all potential problems and to adequately assess the 
situation, it still has to be determined when the resulting knowledge can be called 
"awareness." 


The Definition of "Awareness" 

Adequate situation assessments provide the pilot with valuable knowledge about his 
flight. Still, knowledge does not in any case imply awareness. There is an ongoing debate 
about the question: 


— Is awareness equivalent to actually conscious knowledge (which would 
suggest that only information in primary/working memory should be 
called aware (see e.g., Anderson, 1983)) or does it comprise any 
knowledge than can potentially be retrieved when necessary (as an 
activated subset of long-term memory)? 

This question refers to the distinction between consciousness being a "state" or a "stage" 
(see Klatzky, 1984). 

In the context of situation awareness, it seems to be inappropriate to refer to 
awareness only as information in working memory, i.e., information "currently under 
attentional focus" (see Klatzky, 1984). A definition of situation awareness has to account 
for requirements like the above mentioned temporal aspect. A comprehensive active 
model of the world that is continuously being updated can hardly be stored in short-term 
memory, which is assumed to only hold a very restricted amount of information. Our 
understanding of situation awareness will therefore comprise any information that is 
potentially available and can be activated when relevant in order to assess and cope with 
a situation. 

The practical reason for bothering with situation awareness is the hope to help 
maintain or even increase aviation safety. This hope is based on the assumption of a 
positive relationship between situation awareness and performance. Besides the fact that 
such a relationship has not yet been established, it is important to keep in mind that pilot 
performance is based on both fact and skill knowledge. This also makes it questionable 
to require "awareness of knowledge" in the sense of "conscious knowledge." Theories of 
memory and awareness postulate that only fact knowledge can be(come) conscious 
whereas "procedural knowledge structures [skill knowledge] guiding thought and action 
are unconscious" (Kihlstrom, 1987). The same problem is addressed by the distinction 
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between "explicit" and "implicit" memory. Using this different terminology, Schacter (1987) 
postulates that "implicit memory [skill knowledge] is revealed when previous experiences 
facilitate performance on a task that does not require conscious or intentional recollection 
of those experiences; explicit memory [fact knowledge] is revealed when performance on 
a task requires conscious recollection of previous experiences." 

The above statements implicitly postulate an all-or-none rule of awareness. Cowan 
(1988) suggests that instead of a dichotomy, there may rather be a continuum: "Activation 
must exert a certain threshold before it becomes part of awareness." 

Based on the preceding discussion, our understanding of situation awareness is as 
follows: 


— Situation awareness refers to any knowledge— regardless of its actual level of 
consciousness— that can be activated and adequately processed when required 
in order to cope with a situation. 

This phrase is not meant to be a concise definition of situation awareness. It is far 
too vague for that purpose; however, it can serve as a basis for an operational definition 
of the phenomenon in the context of specific studies. 


The Assessment of Situation Awareness 

The most adequate approach to investigating situation awareness in the aviation 
domain seems to be the staging of complex dynamic scenarios by means of full-mission 
simulation. This approach avoids problems of laboratory studies, for example, simplistic 
tasks and tools or the consideration of isolated awareness components which question the 
validity and applicability of results. Still, it involves difficult issues like the following: 

1. Selection of Situations 

The selection of situations to be staged on the simulator should be 
driven by a predefined phenomenon of interest (like the concept of situation 
awareness) in order to focus in on specific questions rather than generally 
observe pilots’ performance. The challenge is to translate the phenomenon 
from general psychological terms to a domain-specific relevant scenario. At 
this stage, domain experts need to be involved in order to end up with a 
realistic task that is embedded in a credible overall scenario in order to 
motivate pilots and to get valid results. 

2. Adequacy of Assessment Techniques 

Up to now, situation awareness has been investigated by using either 
indirect approaches (e.g., performance measurements) or direct methods 
(e.g., freezing techniques, debriefings; see Endsley, 1989; Marshak et al., 
1987). 
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The problem with performance measures is that a direct relationship between 
performance and awareness has not yet been established. Therefore, it is questionable to 
simply infer the degree of awareness from the pilot’s behavior. Performance measures may 
be the only possible method to assess implicit skill-related memory as it is supposed to 
influence behavior without ever reaching the state of consciousness. Eich (1984) postulates 
that implicit memory can never be revealed in tests that require intentional remembering, 
for example, in a debriefing. Still, performance measures need to be supplemented by 
other techniques. 

The application of direct measures is likely to be intrusive and thereby 
counteracting efforts to be as realistic as possible (e.g., "partial report procedures" 
involving the "blanking of displays" as used by Endsley, 1988). Or it involves after-the-fact 
data collection in a debriefing that may yield misguiding results because it requires 
context-free retrieval of information. Another potential problem of debriefings is that they 
can be used to rationalize or justify behavior during the experimental run rather than 
describe and explain it. 


Promising Alternative Approaches . 

We would like to suggest alternative approaches to the investigation of situation 
awareness. First of all, some modifications with respect to former scenario character and 
implementations may be helpful: 

(1) The investigation of situation awareness requires the introduction of 
unnotified, subtle, slowly developing, and interacting problems instead 
of the usual instant-onset failure that is immediately indicated by an 
alarm. The latter can serve to investigate pilots’ decision-making and 
problem-solving performance. But if situation awareness is the topic 
of interest, it is also important to test the pilot’s ability to realize the 
existence of a problem in the first place. 

(2) The simulation should last for quite a long period of time in order 
to allow for the realistic evolution of a complex situation that can 
serve as a valid touchstone for the pilot’s situation awareness. A 
longer scenario will also provoke vigilance decrements that are likely 
to impair situation awareness and to reveal consequences on 
performance. In addition, it will help to give the pilot a better chance 
to adapt to the simulator. 

(3) In order to guarantee the regular occurrence of all scenario events, 
it will be advantageous to work with a "confederate." In former 
studies, standard scenarios were most often achieved by manipulating 
the situation from outside the simulator. But more complex scenarios 
involving combinations of minor problems with deliberate omission 
and commission errors require the assistance of a "cooperating pilot" 
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who initiates scenario features and who can also assist in probing the 
pilot’s situation awareness. 

The assessment of situation awareness should be carried out during the 
experimental run in order to avoid the above-mentioned problems related to debriefings. 
It will be necessary to develop and test in-flight probing techniques that are not unrealistic 
and intrusive. They have to focus in on knowledge that is relevant for the pilot at the 
time of the probing. Otherwise, there is a risk of erroneously diagnosing "unawareness", 
just because the requested knowledge is, in general, available to the pilot but not 
sufficiently activated at this time. Adequate probing techniques should address different 
information processing stages involved in situation assessments in order to find out at 
which stage problems occur. 

Some candidate techniques are: 

— Effacing specific data from displays when the pilot can be expected 
to pay attention to them instead of totally blanking the display 

— Asking the pilot for previously displayed information by calling him 
as ATC, as his company, or by addressing him as a crew member 

- Having the copilot ("confederate") discuss a problem with the pilot 
in order to find out about his situational knowledge 

— Freezing important indicators 

- Introducing unnotified aircraft in his vicinity that require 
collision-avoidance activities in order to test for environmental 
awareness 

It will be essential to come up with failures, deviations, or errors that require 
immediate, well-defined, corrective actions or information search; otherwise there will be 
the same discussions about the appropriateness and necessity of expected behavior that 
emerged in the context of workload and performance studies. Again, the cooperation with 
domain practitioners will be crucial in order to adequately collect and interpret the 
experimental data. 


Summary and Conclusions 

This paper emphasizes the need for analyzing the cognitive processes underlying 
situation awareness in complex dynamic worlds. We need to identify and understand the 
nature of actually existing problems in order to move beyond the current trial and error 
approach to improving situation awareness. 
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This implies that instead of further discussion of what should be the contents of 
awareness, research should concentrate on the question how to assess and manipulate the 
potential availability of information and knowledge. 

The second part of the paper suggests that full-mission simulation will be the key 
to investigating information processing in complex dynamic worlds, but that traditional 
simulation approaches need to be modified in order to effectively address critical research 
issues. The most important methodological goal is to develop nonintrusive in-flight probing 
techniques to focus in on specific information-processing stages. 


This manuscript was prepared while the authors were supported in part by a grant 
from the NASA-Ames Research Center, Grant No. NCC 2 - 592. 


Air Transport Association of America (1989). National Plan to Enhance Aviation Safety 
through Human Factors Improvements. Washington, D.C.: Author. 

Anderson, J.R. (1983). The architecture of cognition. Cambridge, MA: Harvard University 
Press. 

Chase, W.G. and Simon, H.A. (1973). The mind’s eye in chess. In: W.G. Chase (Ed.). 
Visual information processing. New York: Academic Press. 

Cowan, N. (1988). Evolving Conceptions of Memory Storage, Selective Attention, and 
Their Mutual Constraints Within the Human Information-Processing System. 
Psychological Bulletin, 104(2), 163-191. 

Eich, E. (1984). Memory for unattended events: Remembering with and without 
awareness. Memory \& Cognition, 12(2), 105-111. 

Endsley, M.R. (1988). Situation awareness global assessment technique (SAG AT). Paper 
presented at the National Aerospace and Electronic Conference (NAECON), 
Dayton, OH. 

Endsley, M.R. (1989). A Methodology for the Objective Measurement of Situation 
Awareness. Paper presented at the AGARD Symposium on Situation Awareness 
in Aerospace Operations, Copenhagen, Denmark. 

Fracker, M.L. (1988). A Theory of Situation Assessment: Implications for Measuring 
Situation Awareness. In: Proceedings of the Human Factors Society 32nd Annual 
Meeting (pp. 102-106). Santa Monica, CA: The Human Factors Society. 


32 


Hamilton, W.L. (1987). Situation Awareness Metrics Program. SAE Technical Paper No. 
871 767, paper also presented at the "Aerospace Technology Conference and 
Exhibition", Long Beach, CA. 

Harwood, K., Barnett, B. and Wickens, C. (1988). Situational Awareness: A Conceptual 
and Methodological Framework. In: Proceedings of the 11th symposium of 
psychology in the Department of Defense, April. 

Hayes, J.R., Waterman, D.A. and Robinson, C.S. (1977). Identifying relevant aspects of 
a problem text. Cognitive Science, 1, 1977, 297-313. 

Kihlstrom, J. F. (1987). The Cognitive Unconscious. Science, Vol. 237, 1445-1452. 

Klatzky, R.L. (1984). Memory and Awareness - An Information-Processing Perspective. 
New York: W.H. Freeman and Co. 

Marshak, W.P. et al. (1987). Situational Awareness in Map Displays. In: Proceedings of 
the Human Factors Society 31st Annual Meeting (pp. 533-538). Santa Monica, CA. 
The Human Factors Society. 

McKinnon, F.B. et al. (Eds.)(1986). Final report of the intraflight command, control, and 
communications symposium. 57th Fighter Weapons Wing, Nellis Air Force Base, 
NV. 

National Aeronautics and Space Administration (NASA)(1988). Aviation Safety/Automation 
- A Plan for the Research Initiative. 

Regal, D.M., Rogers, W.H. and Boucek, G.P. (1988). Situational Awareness in the 
Commercial Flight Deck: Definition, Measurement, and Enhancement. SAE 
Technical Paper No. 881 508. Also presented at the Aerospace Technology 
Conference and Exposition, Anaheim, CA, 1988 

Rumelhart, D.E. (1984). Schemata and the Cognitive System. In: R.S. Wyer and T.K. Srull 
(eds.). Handbook of Social Cognition, Vol. 1 (pp. 161-188). Hillsdale, NJ: Lawrence 

Earlbaum. 

Schacter, D. L. (1987). Implicit Memory: History and Current Status. Journal of 
Experimental Psychology: Learning, Memory, and Cognition, 13(3), 501-518. 

Whitaker, L.A. and Klein, G.A. (1988). Situation awareness in the virtual world: Situation 
assessment report. In: Proceedings of the 11th symposium of psychology in the 
Department of Defense, April. 

Wickens, C.D. (1984). Engineering Psychology and Human Performance. Columbus, OH: 
Charles E. Merril. 


33 



Wiener, E. L. (1989). Human Factors of Advanced Technology ("Glass Cockpit") 
Transport Aircraft. NASA Contractor Report 177 528, May. 


34 


APPENDIX B 


BIBLIOGRAPHY: SITUATION AWARENESS 


Compiled by 
J. R. Comstock, Jr. 

NASA Langley Research Center 


Abbott, T. S. Simulation of a cockpit display concept for increased airport capacity. 
Human Factors Society, Annual Meeting, 27th, Norfolk, VA, October 10-14, 1983, 
Proceedings. Santa Monica, CA, Human Factors Society, 1983, p. 777-781. 

Abbott, T. S., Moen, G. C., Person, L. H., Jr., Keyser, G. L., Jr., Yenni, K. R., and 
Garren, J. F., Jr. Early flight test experience with Cockpit Displayed Traffic 
Information (CDTI). NASA TM-80221, National Aeronautics and Space 
Administration. Langley Research Center, Hampton, VA. 

Abbott, T. S., Moen, G. C., Person, L. H., Jr., Keyser, G. L., Jr., Yenni, K. R., and 
Garren, J. F., Jr. Flight investigation of cockpit-displayed traffic information 
utilizing coded symbology in an advanced operational environment. NASA TP- 1684, 
National Aeronautics and Space Administration. Langley Research Center, 
Hampton, VA. 

Adams, R. J., and Thompson, J. L. Aeronautical decision making for air ambulance 
helicopter pilots: learning from past mistakes. DOT/FAA/DS-88/5, TLSP: Final 
Report, Systems Control Technology, Inc., Arlington, VA. 

Adams, R. J., and Thompson, J. L. Aeronautical decision making for air ambulance 
helicopter pilots: Situational awareness exercises. DOT/FAA/DS-88/6, TLSP: Final 
Report, Systems Control Technology, Inc., Arlington, VA. 

Adams, R. J., and Thompson, J. L. Risk management for air ambulance helicopters 
operators. DOT/FAA/DS-88/7, TLSP: Final Report, Systems Control Technology, 
Inc., Arlington, VA. 

Alkov, Robert A. Human error mishap causation in naval aviation. SAE paper 872508. 
Human Error Avoidance Techniques Conference, Washington, DC, Dec. 1-3, 1987, 
Proceedings. Warrendale, PA, Society of Automotive Engineers, Inc., 1988, p. 23- 
26. 

Arbak, C. J., Schwartz, N., and Kuperman, G. Evaluating the Panoramic Cockpit Controls 
and Displays System. Proceedings of the Fourth International Symposium on 
Aviation Psychology, Columbus, OH, Apr. 27-30, 1987. Ohio State University, 
Columbus, OH, 1987, p. 30-36. 


35 



Aretz, A. J, A model of electronic map interpretation. Proceedings of the Human 
Factors Society 32nd Annual Meeting, Anaheim, CA, Oct. 24-28, 1988, Santa 
Monica, CA, Human Factors Society, 1988, p. 130-134. 

Aretz, A. J., Reising, J. M., Kopala, C. J., Calhoun, G. L., and Herron, E. L. Computer 
generated pictorial stores management displays for fighter aircraft. AD-A131412, 
TLSP: Final Report, 1 Aug. 1980 - 30 Sep. 1981. Air Force Wright Aeronautical 
Labs., Wright-Patterson AFB, OH. 

Bemis, S., I.eeds, J., and Winer, E. Operator performance as a function of type of 
display: conventional vs. perspective. Human Factors, 30, 163-169, 1988. 

Beringer, D. B., and Hancock, P. A. Exploring situational awareness: A review and the 
effects of stress on rectilinear normalization. Proceedings of the 5th International 
Symposium on Aviation Psychology (1989). 

Biferno, M. A., and Stanley, D. L. The touch-sensitive control/ display unit - A promising 
computer interface. SAE paper 831532. Aerospace Behavioral Engineering 
Technology Conference, 2nd, Long Beach, CA, October 3-6, 1983, Proceedings. 
Warrendale, PA, Society of Automotive Engineers, Inc., 1983, p. 409-413. 

Calhoun, G. L., Valencia, G., and Furness, T. A., Ill Three-dimensional auditory cue 
simulation for crew station design/ evaluation. Proceedings of the 31st Annual 
Meeting of the Human Factors Society, Santa Monica, CA, 1987, p. 1398-1402. 

Carr, L. A., and Lopina, R. F. Pilot workload in the night attack mission. NAECON 
1982; Proceedings of the National Aerospace and Electronics Conference, Dayton, 
OH, May 18-20, 1982. New York, Institute of Electrical and Electronics Engineers, 
Inc., 1982, p. 881-884. 

Cavanagh, D. E., and Williams, K. R. The application of CRM to military operations. 
In NASA: Ames Research Center Cockpit Resource Management Training, p. 135- 
144. 

De Decker, R. W. Situational awareness. Vertical flight training needs and solutions; 
Proceedings of the AHS National Specialists’ Meeting, Arlington, TX, Sept. 17, 18, 
1987. Alexandria, VA, American Helicopter Society, Inc., 1987, p. 27-31. 


Dornheim, M. A. Crew situational awareness drives avionics developments. Aviation 
Week and Space Technology, vol. 124, June 23, 1986, p. 114-116. 

Eisenhardt, R. G., Eisenhardt, G. H., Douthat, D. Z. Pilot’s associate definition study. 
AD-A157106, TLSP: Final Report, Jul. 1984 - May 1985. Perceptronics, Inc., Ann 
Arbor, MI. 

Ellis, S., McGreevy, M., and Hitchcock, R., Perspective traffic display format and airline 
pilot traffic avoidance. Human Factors, 29, 371-382, 1987. 


36 



Emerson, T. J., Reising, J. M., and Britten-Austin, H. G. Workload and situation 
awareness in future aircraft. Recent advances in cockpit aids for military 
operations; Proceedings of the Symposium, London, England, Mar. 31, 1987. 
London, Royal Aeronautical Society, 1987, p. 33-44. 

Emerson, T. J., Reising, J. M., and Britten-Austin, H. G. Workload and situation 
awareness in future aircraft. SAE Paper 871803, Aerospace Behavioral Engineering 
Technology Conference, 6th, Long Beach, CA, Oct. 5-8, 1987, Proceedings. 
Warrendale, PA, Society of Automotive Engineers, Inc., 1988, p. 107-114. 

Endsley, M. R. The application of human factors to the development of expert systems 
for advanced cockpits. Proceedings of the 31st Annual Meeting of the Human 
Factors Society, New York, NY, Oct. 19-23, 1987. Human Factors Society, Santa 
Monica, CA, 1987, p. 1388-1392. 

Endsley, M. R. Situation awareness global assessment technique (SAGAT) — aircraft 
pilots-vehicle interface design. NAECON 88; Proceedings of the IEEE National 
Aerospace and Electronics Conference, Dayton, OH, May 23-27, 1988. New York, 
Institute of Electrical and Electronics Engineers, 1988, p. 789-795. 

Endsley, M. R. Design and evaluation for situation awareness enhancement. Proceedings 
of the 32nd Annual Meeting of the Human Factors Society, Anaheim, CA, Oct. 24- 
28, Human Factors Society, Santa Monica, CA, 1988, p. 97-101. 

Fracker, M. L. A theory of situation assessment - Implications for measuring situation 
awareness. Proceedings of the 32nd Annual Meeting of the Human Factors 
Society, Anaheim, CA, Oct. 24-28, 1988. Human Factors Society, Santa Monica, 
CA, 1988, p. 102-106. 

Fracker, M. L. A schema-based model of situation awareness: Implications for measuring 
situation awareness. Second Annual Workshop on Space Operations Automation 
and Robotics (SOAR 1988). NASA CP-3019, p. 227-231. 

Griffin, G. R., Morrison, T. R., Amerson, T. L., and Hamilton, P. V. Predicting Air 
Combat Maneuvering (ACM) performance: Fleet fighter ACM readiness program 
grades as performance criteria. AD-A191605, TLSP: Interim Report, 1986 - 1987 

Griffin, R. Predicting Air Combat Maneuvering (ACM) performance: Fleet fighter ACM 
readiness program grades as performance criteria. AD-A203530, TLSP: Interim 
Report, Naval Aerospace Medical Research Lab., Pensacola, FL. 

Guttman, J. Evaluation of the F/A-18 head-up-display for recovery from unusual attitudes. 
AD- A 183303, TLSP: Final Report Naval Air Development Center, Warminster, PA. 

Hamilton, W. L. Situation awareness metrics program — in fighter cockpits. SAE Paper 
871767, SAE Aerospace Technology Conference and Exposition, Long Beach, CA, 
Oct. 5-8, 1987. 5 p. 


37 



Hanson, R. F. Situation awareness and the PVI link - Pilot-Vehicle Interface. AIAA 
PAPER 88-3885, AIAA/IEEE Digital Avionics Systems Conference, 8th, San Jose, 
CA, Oct. 17-20, 1988. American Institute of Aeronautics and Astronautics, 
Washington, DC, 1988, p. 193-195. 

Hart, S. G., and Chappell, S. L. Influence of pilot workload and traffic information on 
pilot’s situation awareness. 19th Annual Conference on Manual Control, 
Massachusetts Inst, of Tech., p. 4-26. 

Harwood, K., Barnett, B., and Wickens, C. Situational awareness: A conceptual and 
methodological framework. Proceedings of the Eleventh Symposium on Psychology 
in the Department of Defense. Colorado Springs: USAF Academy, 321-325. 

Henderson, E. D. Air traffic controller awareness and resource training. SAE Paper 
881518, 7th Aerospace Behavioral Technology Conference and Exposition, Anaheim, 
CA, Oct. 3-6, 1988, Proceedings. Society of Automotive Engineers, Inc., 
Warrendale, PA, 1989, p. 141-144. 

Hinton, D. A. A flight test evaluation of the pilot interface with a digital advanced 
avionics system. AIAA Paper 84-1819, Ajnerican Institute of Aeronautics and 
Astronautics, Guidance and Control Conference, Seattle, WA, Aug. 20-22, 1984. 
9 p. 

Hinton, D. A. A simulation evaluation of a pilot interface with an automatic terminal 
approach system. NASA-TP-2669, National Aeronautics and Space Administration. 
Langley Research Center, Hampton, VA. 

Houck, R., Kelly, B. D., and Wiedemann, J. An integrated display for vertical situation 
awareness in commercial transport aircraft. SAE Paper 861770, 5th Aerospace 
Behavioral Engineering Technology Conference, Long Beach, CA, Oct. 13-16, 1986, 
Proceedings. Society of Automotive Engineers, Inc., Warrendale, PA, 1986, p. 61- 
64. 

Houck, R. D., Rogers, W. H., and Braune, R. J. Advanced technology cockpit design and 
the management of human error. Human Error Avoidance Techniques Conference, 
Washington, DC, Dec. 1-3, 1987. Society of Automotive Engineers, Inc., 
Warrendale, PA, 1988, p. 71-77. 

Hynes, C. S., Franklin, J. A., Hardy, G. H., Martin, J. L., and Innis, R. C. Flight 
evaluation of pursuit displays for precision approach of powered-lift aircraft. 
Journal of Guidance, Control, and Dynamics, 12, Jul-Aug. 1989, 521-529. 

Kalawsky, R. S. Pilot integration and the implications on the design of advanced cockpits. 
In AGARD, The Man-Machine Interface in Tactical Aircraft Design and Combat 
Automation. 16 p. 


38 


King, P. A. Big Picture: A solution to the problem of situation awareness - in military 
aircraft. NAECON 87; Proceedings of the IEEE National Aerospace and 
Electronics Conference, Dayton, OH, May 18-22, 1987. New York, Institute of 
Electrical and Electronics Engineers, Inc., 1987, p. 261-265. 

Kuperman, G. G., and Wilson, D. L. The design of a tactical situation display. 
Proceedings of the 32nd Annual Meeting of the Human Factors Society, Anaheim, 
CA, Oct. 24-28, 1988. Human Factors Society, Santa Monica, CA, 1988, p. 111-115. 

Marshak, W. P., Kuperman, G., Wilson, D., and Ramsey, E. G. Situational awareness in 
map displays. Proceedings of the 31st Annual Meeting of the Human Factors 
Society, New York, NY, Oct. 19-23, 1987. Human Factors Society, Santa Monica, 
CA, 1987, p. 533-535. 

McNaughton, G. B. Vision in spatial disorientation (SDO) and loss of situational 
awareness. Symposium on Aviation Psychology, 3rd, Columbus, OH, April 22-25, 
1985. Ohio State University, Columbus, OH 1985, p. 25-38. 

Nataupsky, M., and Crittenden, L. Stereo 3-D and non-stereo presentations of a 
computer-generated pictorial primary flight display with pathway augmentation. 
AIAA PAPER 88-3965, AIAA/IEEE Digital Avionics Systems Conference, 8th, San 
Jose, CA, Oct. 17-20, 1988. Washington, DC, American Institute of Aeronautics and 
Astronautics, 1988, p. 552-557. 

Norman, R. E., Jr. Training pilots in the area of judgment, decision making and cockpit 
management. SAE PAPER 831498, Aerospace Behavioral Engineering Technology 
Conference, 2nd, Long Beach, CA, October 3-6, 1983. Society of Automobile 
Engineers, Inc., Warrendale, PA, 1983, p. 311-313. 

Paden, W. R., Jr. Flight decks of the future - Technology explodes. Lockheed Horizons, 
No. 13, 1983, p. 18-23. 

Palko, K., and Boys, R. Augmenting the pilot/vehicle interface through capability 
assessment. NAECON 87; Proceedings of the IEEE National Aerospace and 
Electronics Conference, Dayton, OH, May 18-22, 1987. New York, Institute of 
Electrical and Electronics Engineers, Inc., 1987, p. 956-964. 

Palmer, M. T., Abbott, K. H., Schutte, Paul C., and Ricks, W. R. Implementation of a 
research prototype onboard fault monitoring and diagnosis system. AIAA PAPER 
87-2777, AIAA Computers in Aerospace Conference, 6th, Wakefield, MA, Oct. 7-9, 
1987. Washington, DC, American Institute of Aeronautics and Astronautics, 1987, 
p. 100-107. 

Person, L. H., Jr., and Steinmetz, G. G. The integration of control and display concepts 
for improved pilot situational awareness. Flight Safety Foundation, 34th 
International Air Safety Seminar, Acapulco, Mexico, Nov. 9-12, 1981. 


39 


Regal, D. M., Rogers, W. H., and Boucek, G. P., Jr. Situational awareness in the 
commercial flight deck - Definition, measurement, and enhancement. SAE Paper 
881508, Aerospace Behavioral Technology Conference and Exposition, 7th, 
Anaheim, CA, Oct. 3-6, 1988. Society of Automotive Engineers, Inc., Warrendale, 
PA, 1989, p. 65-69. 

Rouse, D. M., and Hummel, T. C. The Pilot’s Associate - Enhancing situational 
awareness through cooperating expert systems. SAE Paper 871896. Aerospace 
Behavioral Engineering Technology Conference, 6th, Long Beach, CA, Oct. 5-8, 
1987. Society of Automotive Engineers, Inc., Warrendale, PA, 1988, p. 121-125. 

Steinmetz, G. G. Development and evaluation of an airplane electronic display format 
aligned with the inertial velocity vector. NASA-TP-2648, National Aeronautics and 
Space Administration. Langley Research Center, Hampton, VA. 

Storey, B. A. Simulation tools for crew system assessment. AIAA PAPER 88-4614, 
AIAA Flight Simulation Technologies Conference, Atlanta, GA, Sept. 7-9, 1988, 
Technical Papers (A88-53626 23-09). Washington, DC, American Institute of 
Aeronautics and Astronautics, 1988, p. 226-229. 

Weber, J. W. Future electronic map systems. NAECON 1984; Proceedings of the 
National Aerospace and Electronics Conference, Dayton, OH, May 21-25, 1984. 
New York, IEEE, 1984, p. 106-111. 

Whitehead, A. M., Thomas, W. P., and Wilkins, M. P. Integrated flying aid and mission 
displays for modern combat aircraft incorporating a digital data base. Digital 
Avionics Systems Conference, 7th, Fort Worth, TX, Oct. 13-16, 1986. New York, 
Institute of Electrical and Electronics Engineers, Inc., 1986, p. 267-274. 

Wickens, C. D., Aretz, A., and Harwood, K. Frame of references for electronic maps: 
The relevance of spatial cognition and componential task analysis. Proceedings of 
the 5th International Symposium on Aviation Psychology, 1989. 


40 


APPENDIX C 


AVIATION SAFETY/AUTOMATION PROGRAM WORKSHOP 
LIST OF PARTICIPANTS 


Ms. Kathy H. Abbott 

NASA Langley Research Center 

MS 156A 

Hampton, VA 23665-5225 
804/864-2018 

Dr. Sheldon Baron 

BBN Systems & Technologies Corp. 

70 Fawcett St. 

Cambridge, MA 02138 
617/873-3235 

Dr. Michael A. Biferno 
Douglas Aircraft Co. 

Mail Code 78-73 
3855 Lakewood Blvd. 

Long Beach, CA 90846 
213/593-7094 

Dr. Thomas Chidester 
NASA Ames Research Center 
MS 239-15 

Moffett Field, CA 94035 

Dr. J. Raymond Comstock 
NASA Langley Research Center 
MS 152E 

Hampton, VA 23665-5225 
804/864-6643 

Dr. Kevin M. Corker 

BBN Systems & Technologies Corp. 

70 Fawcett St. 

Cambridge, MA 02138 

Dr. William Corwin 
Douglas Aircraft Co. 

Mail Code 78-73 
3855 Lakewood Blvd. 

Long Beach, CA 90846 
213/593-9047 


Dr. R. K. Dismukes 
NASA Ames Research Center 
MS 239-1 

Moffett Field, CA 94035 
415/694-5729 

Dr. R. Curtis Graeber 
NASA Ames Research Center 
MS 239-21 

Moffett Field, CA 94035 
415/694-5792 

Dr. Randall L. Harris, Sr. 

NASA Langley Research Center 
MS 152E 

Hampton, VA 23665-5225 
804/854-6641 

Ms. Sandra G. Hart 
NASA Ames Research Center 
MS 239-5 

Moffett Field, CA 94035 
Ms. Eva Hudlicka 

BBN Systems & Technologies Corp. 
70 Fawcett St. 

Cambridge, MA 02238 

Dr. James P. Jenkins 
NASA HQ 
Attn: RC 

Washington, DC 20546 
202/453-2750 

Mr. Gary D. Lium 
FAA 

Aircraft Certification Division 
17900 Pacific Hwy. S 
C-68966 ANM-111 
Seattle, WA 98168 


41 



Mr. William L. Miles 
Douglas Aircraft Co. 

Mail Code 78-73 
3855 Uikcwood Blvd. 

Long Beach, CA 90846 
213/593-8168 

Mr. Samuel A. Morello 
NASA Langley Research Center 
MS 153 

Hampton, VA 23666-5225 
804/864-6664 

Dr. Everett A. Palmer 
NASA Ames Research Center 
MS 239-21 

Moffett Field, CA 94035 
415/694-6073 

Mr. Michael T. Palmer 
NASA Langley Research Center 
MS 156A 

Hampton, VA 23665-5225 
804/864-2044 

Dr. Raja Parasuraman 
Catholic University of America 
Department of Psychology 
Washington, DC 20064 
202/635-5750 

Mr. Dick Pew 

BBN Systems & Technologies Corp. 
70 Fawcett St. 

Cambridge, MA 02138 

Dr. Alan Pone 

NASA Ixingley Research Center 
MS 152E 

Hampton, VA 23665-5225 
804/864-6642 

Mr. Wendell R. Ricks 
NASA Langley Research Center 
MS 156A 

Hampton, VA 23665-5225 
804/864-6733 

Dr. William H. Rogers 
Boeing Commercial Airplane Co. 
2514 186th Ave. NE 
Redmond, WA 98052 
206/237-7287 


Mr. Loren Rosenthal 

Battelle 

Battelle Blvd. 

Richland, WA 99352 
509/376-2914 

Dr. Renate Roske-Hofstrand 
NASA Ames Research Center 
MS 239-21 

Moffett Field, CA 94035 
415/694-6911 

Dr. William Rouse 
Search Technology, Inc. 

4725 Peachtree Corners Cir. 

Suite 200 

Norcross, GA 30092 

Ms. Nadine B. Sarter 

Ohio State University 

Dept, of Industrial <x Systems 

Engineering 

290 Baker Hall 

1971 Neil Ave. 

Columbus, OH 43210 
614/292-6287 

Mr. Paul Schutte 

NASA Langley Research Center 

MS 156A 

Hampton, VA 23665-5225 
804/864-2019 

Mr. George G. Steinmetz 
NASA Langley Research Center 
MS 156 A 

Hampton, VA 23665-5225 
804/864-3844 

Mr. Harty Stoll 
Mail Code 77-35 

Boeing Commercial Airplane Co. 

P.O. Box 3707 

Seattle, WA 98124-2207 

Dr. David Woods 

Industrial & Systems Engineering 

290 Baker Hall 

Ohio State University 

1971 Neil Ave. 

Columbus, OH 43210 


42 



Report Documentation Page 

Scxe Acy^svaion 

1. Report No. 

NASA CP- 10054 

2. Government Accession No. 

3. Recipient's Catalog No. 

4. Title and Subtitle 

Report of the Workshop on Aviation Safety/ 
Automation Program 

5. Report Date 

October 1 990 

6. Performing Organization Code 

7. Author(s) 

Samuel A. Morello (Editor) 

8. Performing Organization Report No. 

10. Work Unit No. 

505-64-13-22 

9. Performing Organization Name and Address 

NASA Langley Research Center 
Hampton, VA 23665-5225 

11. Contract or Grant No 

13. Type of Report and Period Covered 

Conference Publication 

12. Sponsoring Agency Name and Address 

National Aeronautics and Space Administration 
Washington, DC 20546 

14. Sponsoring Agency Code 

15. Supplementary Notes 

16 #§ ,r j5Srt of NASA's responsibility to encourage and facilitate active exchange of 
information and ideas among members of the aviation community, an Aviation 
Safety/Automation workshop was organized and sponsored by the Flight 
Management Division of NASA Langley Research Center. The one-day workshop 
was held on October 10, 1989, at the Sheraton Beach Inn and Conference Center 
in Virginia Beach, Virginia. Participants were invited from industry, Government, 
and universities to discuss critical questions and issues concerning the rapid 
introduction and utilization of advanced computer-based technology into the flight 
deck and air traffic controller workstation environments. The workshop was 
attended by approximately 30 discipline experts, automation and human factors 
researchers, and research and development managers. The goal of the workshop 
was to address major issues identified by the NASA Aviation Safety/Automation 
Program. This report documents the results of the workshop. The ideas, fbbugbts, 
and concepts were developed by the workshop participants. The findings, 
however, have been synthesized into a final report primarily by the NASA 
researchers. 

17. Key Words (Suggested by Author(sJ) 

Av Safety 
Human Factors 
Situation Awareness 
Technology Transfer 

18. Distribution Statement 

Unclassified - Unlimited 

Subject Category: 03 

19. Security Classif. (of this report) 

Unclassified 

20. Security Classif. (of this page) 

Unclassified 

21. No. of pages 22. Price 

49 A0 3 


NASA FORM 1626 OCT 86 


For sale by the National Technical Information Service, Springfield, Virginia 22161-2171 











